Servers in distributed computer systems are frequently relied on to schedule tasks for processing requests from a large number of clients. However, in situations where there are a large number of tasks to be performed, a server can quickly become overwhelmed by the process of scheduling the tasks for execution. Conventional approaches to task scheduling can result in substantial wasted resources either because too many resources are dedicated to a small number of tasks or because not enough resources are dedicated to a large number of tasks. Indeed, the same server may face both of these problems sequentially or in parallel due to the fact that the rate at which requests are received from clients can vary dramatically over a typical operational time period. Moreover, a server that is inefficient or overwhelmed can become a bottleneck for task processing and consequently increase the latency of responses to the client requests. These and other problems with conventional approaches to task scheduling are reduced or eliminated by the systems and methods described below.